The present application relates to an image processing method, an image processing apparatus, a program and a recording medium, respectively for making moving image data having a high resolution.
In the field of image processing, one main idea is to improve image resolution. Technologies for creating a high resolution image by enlarging one low resolution image such as through pixel interpolation have been studied. There is a limit, however, in creating a high or super resolution image from a low resolution image because the frequency band of a signal of the low resolution image is limited.
High resolution or super resolution technologies are well known by which a super resolution image having a large number of pixels is created from a plurality of low resolution images. A variety of methods for super resolution analysis have been proposed. Well known methods include a method using a frequency space (refer to “Multiframe image restoration and registration”, by R. Y. Tsai and T. S. Huang, Advances in Computer Vision and Image Processing, vol. 1, JAIPress Inc., 1984), a method based on MAP estimation (refer to “Extraction of high-resolution frames from video sequences”, by R. R. Shultz and R. L. Stevenson, IEEE transactions on Image Processing, Vol. 5, No. 6, June 1996), a method by Projection on Convex Sets (refer to “High-resolution image reconstruction from lower-resolution image sequences and space-varying image restoration”, by A. M. Tekalp, M. K. Ozkan and M. I. Sezan, Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), San Francisco, Calif., vol. 3, March 1992), and the like.
Although the above-described methods in related art, particularly the method based on MAP estimation and the like, can obtain very good results, these methods require a number of input images, resulting in an issue of a very high calculation cost. There is another issue that motion compensation for object blurring is difficult.
Many super resolution technologies assume that a plurality of input low resolution images are obtained by photographing the same scene through stepwise parallel displacement of a camera. There arises, therefore, an issue of object blurring if each individual object moves in a scene, if a camera is moved in a scene having a depth, or in other cases.
Although the technology by R. R. Shultz, et al deals with object blurring, it is necessary to estimate a motion vector of an input image at each pixel position, resulting in an issue of a high calculation cost. The issue associated with this technology is that even if a motion vector is detected from a low resolution input image, a motion vector cannot be estimated correctly because of noises, aliasing and the like.
Accordingly, it is desirable to provide an image processing method, an image processing apparatus, a program or a recording medium, respectively being capable of generating super resolution moving image data from low resolution moving image data with a calculation amount smaller than that in the related art. The present invention has been made in view of the issues existing in the above-described related art.